learn incoher spars low drank pattern multipl task problem learn incoher spars lowrank pattern multipl task approach base linear multi task learn formul spars low rank pattern induc cardin regular term lowrank constraint formul convex convert convex surrog routin solv semidefinit program small size problem propos emploi gener project gradient scheme effici solv convex surrog optim formul object function differenti feasibl domain trivial present procedur comput project gradient ensur global converg project gradient scheme comput project gradient involv constrain optim problem show optim solut problem obtain solv unconstrain optim subproblem euclidean project subproblem addit present project gradient algorithm discuss rate converg experiment result benchmark data set demonstr effect propos multi task learn formul effici propos project gradient algorithm scalabl dstage approach class dimension reduct techniqu dimension reduct plai import role data mine applic involv high dimension data exist dimension reduct techniqu formul gener eigenvalu problem scale larg size problem prior work transform gener eigenvalu problem equival squar formul solv effici equival relationship hold assumpt regular sever limit applic practic paper effici stage approach propos solv class dimension reduct techniqu includ canon correl effici algorithm class fuse lasso problem multipl structur align consensu identif protein fast overlap group lasso group lasso extens lasso featur select predefin overlap group featur overlap group structur limit applic practic recent attempt studi gener formul group featur potenti overlap group result optim challeng solv due group overlap paper effici optim overlap group lasso penal problem reveal kei properti proxim oper overlap group lasso comput proxim oper solv smooth andconvexdualproblem whichallow theuseofthegradientdescenttyp algorithm optim perform empir evalu breast cancer gene express data set consist gene organ overlap gene set experiment result demonstr effici effect propos algorithm effici flq norm regular share dsubspac learn framework multi dlabel classif multi label problem aris domain multi topic document categor protein function predict automat imag annot natur deal problem construct binari classifi label result set independ binari classif problem multipl label share input space semant convei label correl essenti exploit correl inform contain label paper gener framework extract share structur multi label classif framework common subspac assum share multipl label show optim solut propos formul obtain solv gener eigenvalu problem problem nonconvex high dimension problem direct comput solut expens develop effici algorithm case appeal featur propos framework includ algorithm special case elucid intrins relationship show propos framework extend kernel induc featur space conduct extens experi multi topic web page categor automat gene express pattern imag annot task result demonstr squar formul class gener eigenvalu problem machin learn machin learn algorithm formul gener eigenvalu problem major limit formul gener eigenvalu problem computation expens solv larg scale problem paper show mild condit class gener eigenvalu problem machin learn formul squar problem class problem includ classic techniqu dmonoton featur select problem select subset inform featur number requir featur featur select problem essenti combinatori optim problem solv approxim convent featur select method address comput challeng step rank featur score comput independ number featur select top rank featur major shortcom approach featur chosen number featur chosen number featur larger refer properti monoton properti featur select work argu import develop effici algorithm monoton featur select end develop algorithm monoton featur select approxim relat combinatori optim problem multipl kernel learn mkl problem present strategi deriv discret solut approxim solut mkl show perform guarante deriv discret solut compar global optim solut relat combinatori optim problem empir studi number benchmark data set promis onlin learn ellipsoid method work extend ellipsoid method origin design convex optim onlin learn kei idea approxim ellipsoid classif hypothes consist train exampl receiv contrast onlin learn algorithm singl classifi maintain iter effici algorithm present updat centroid posit definit matrix ellipsoid misclassifi addit classic ellipsoid method improv version onlin learn present mistak bound ellipsoid method deriv evalu usp dataset uci data set show encourag result compar propos onlin learn algorithm state art onlin learner convex formul learn share structur multipl task multi task learn mtl aim improv gener perform learn multipl relat task simultan paper problem learn share structur multipl relat task present improv formul iaso multi task learn base convex altern structur optim aso algorithm task relat share featur represent convert iaso convex formul relax convex scalabl larg data set due complex constraint propos altern optim caso algorithm solv convex relax effici show caso converg global optimum addit present theoret condit caso find global optim solut iaso experi benchmark data set confirm theoret analysi acceler gradient method trace norm minim minim smooth loss function regular trace norm matrix variabl formul find applic machin learn task includ multi task learn matrix classif matrix complet standard semidefinit program formul problem computation expens addit due smooth natur trace norm optim order black box method solv class problem converg iter counter paper exploit special structur trace norm base propos extend gradient algorithm converg propos acceler gradient algorithm achiev optim converg rate smooth problem experi multi task learn problem demonstr effici propos algorithm effici euclidean project linear time problem comput euclidean project vector length close convex set includ ball special polyhedra emploi shalev shwartz amp singer problem plai build block role solv norm base spars learn problem exist method worst case time complex log paper propos cast euclidean project root find problem specif auxiliari function solv linear time bisect make special structur auxiliari function propos improv bisect algorithm empir studi demonstr propos algorithm effici compet comput project linear dimension reduct multi dlabel classif dimension reduct essenti step high dimension data analysi dimension reduct algorithm appli successfulli multi class multi label problem commonli appli separ data preprocess step classif algorithm paper studi joint learn framework perform dimension reduct multi label classif simultan show squar loss classif joint learn decoupl separ compon dimension reduct multi label classif analysi partial justifi current practic separ applic dimension reduct classif problem extend analysi loss function includ hing loss squar hing loss extend formul gener case input data class label differ overcom limit tradit dimension reduct algorithm experi benchmark data set conduct evalu propos joint formul learn optim neighborhood kernel classif kernel method appli successfulli applic kernel matrix plai import role kernel base learn method ideal kernel matrix unknown practic estim paper propos directli learn ideal kernel matrix call optim neighborhood kernel matrix pre kernel matrix improv classif perform assum prespecifi kernel matrix gener specif applic noisi observ ideal result optim neighborhood kernel matrix shown summat pre kernel matrix rank matrix formul problem learn optim neighborhood kernel constrain quartic problem propos solv method level method constrain gradient descent empir result benchmark data set demonstr effici effect propos algorithm equival canon correl analysi orthonorm partial squar canon correl analysi cca partial squar pl techniqu featur extract set multidimension variabl fundament differ cca pl cca maxim correl pl maxim covari cca pl appli successfulli applic intrins relationship remain unclear paper attempt address issu show equival relationship cca orthonorm partial squar opl variant pl extend equival relationship case regular emploi set variabl addit show cca project set variabl independ regular set variabl perform experiment studi synthet real data set result confirm establish equival relationship present analysi insight connect exist algorithm effect regular multipl kernel learn multipl label classif multipl label common approach learn classifi label kernel base classifi option set kernel select specif kernel label kernel label work present unifi framework multi label multipl kernel learn approach consid extrem case framework kernel share partial multipl label enabl flexibl degre label common systemat studi share kernel multipl label affect perform base extens experi benchmark data includ imag microarrai data interest find efficaci effici report multiclass probabilist kernel discrimin analysi kernel discrimin analysi kda effect approach supervis nonlinear dimension reduct probabilist model kda improv robust state art model handl binari class problem confin applic real world problem overcom limit propos nonparametr probabilist model base gaussian process kda handl multiclass problem model bayesian interpret kda paramet automat tune optim margin loglikelihood data empir studi demonstr efficaci propos model drosophilagen express pattern annot multi dinstanc multi dlabel learn berkelei drosophila genom project bdgp produc larg number gene express pattern annot textual anatom development term term spatial correspond local region imag attach collect group imag unknown term assign region imag group pose challeng develop comput method autom textual descript express pattern contain imag paper show underli natur task match machin learn framework multi instanc multi label learn miml propos miml support vector machin solv problem beset annot task empir studi show propos method outperform state art drosophila gene express pattern annot method drosophila gene express pattern annot spars featur term dterm interact larg dscale spars logist regress logist regress classif method wide applic data mine machin learn comput vision bioinformat spars logist regress emb featur select classif framework norm regular attract applic involv high dimension data paper propos lassplor solv larg scale spars logist regress specif formul problem ball constrain smooth convex optim propos solv problem nesterov method optim order black box method smooth convex optim critic issu nesterov method estim step size optim iter previou approach appli constant step size assum lipschitz gradient advanc requir sequenc decreas step size lead slow converg practic paper propos adapt line search scheme tune step size adapt guarante optim converg rate empir comparison state theart algorithm demonstr effici propos lassplor algorithm larg scale problem mine discret pattern binari matrix factor mine discret pattern binari data import subsampl compress cluster rankon binari matrix approxim identifi domin pattern data preserv discret properti approxim data minimum set inconsist entri mismatch binari data approxim matrix due hard problem previou account problem emploi heurist result approxim optim paper show rank binari matrix approxim reformul integ linear program ilp ilp formul computation expens small size matric propos linear program lp relax shown achiev guarante approxim error bound extend propos formul regular techniqu commonli emploi address overfit lp formul restrict medium size matric due larg number variabl involv larg matric interestingli show propos approxim formul transform instanc minimum cut problem solv effici find maximum flow empir studi show effici propos algorithm base maximum flow result confirm establish theoret bound bag dof dword approach drosophila gene express pattern annot small sphere larg margin approach novelti detect train data outlier compress facad displac map sixth intern confer machin learn applic train svm indefinit kernel similar matric gener applic posit semidefinit fit kernel machin framework paper studi problem train support vector machin indefinit kernel regular svm formul indefinit kernel matrix treat noisi observ unknown posit semidefinit proxi kernel support vector proxi kernel comput simultan propos semi infinit quadrat constrain linear program formul optim solv iter find global optimum solut propos emploi addit prune strategi significantli improv effici algorithm retain converg properti algorithm addit show close relationship propos formul multipl kernel learn experi collect benchmark data set demonstr effici effect propos algorithm heterogen data fusion alzheim diseas studi learn subspac kernel classif kernel method appli successfulli data mine task subspac kernel learn recent propos discov effect low dimension subspac kernel featur space improv classif paper propos construct subspac kernel hilbert schmidt independ criterion hsic show optim subspac kernel obtain effici solv eigenvalu problem limit exist subspac kernel learn formul kernel learn classif independ subspac kernel optim adapt classif overcom limit propos joint optim framework learn subspac kernel subsequ classifi simultan addit propos learn formul extract uncorrel subspac kernel reduc redund inform subspac kernel idea multipl kernel learn extend propos formul case multipl kernel combin show integr subspac kernel formul semidefinit program sdp computation expens improv effici sdp formul propos equival semi infinit linear program silp formul solv effici column gener techniqu experiment result collect benchmark data set demonstr effect propos algorithm extract share subspac multi dlabel classif multi label problem aris domain multitop document categor protein function predict natur deal problem construct binari classifi label result set independ binari classif problem multipl label share input space semant convei label correl essenti exploit correl inform contain label paper gener framework extract share structur multi label classif framework common subspac assum share multipl label show optim solut propos formul obtain solv gener eigenvalu problem problem nonconvex high dimension problem direct comput solut expens develop effici algorithm case appeal featur propos framework includ algorithm special case elucid intrins relationship conduct extens experi eleven multitop web page categor task result demonstr effect propos formul comparison repres algorithm hypergraph spectral learn multi dlabel classif hypergraph gener tradit graph edg arbitrari empti subset vertex set appli successfulli captur highord relat domain paper propos hypergraph spectral learn formul multi label classif hypergraph construct exploit correl inform label show propos formul lead eigenvalu problem computation expens larg scale problem reduc comput cost propos approxim formul shown equival squar problem mild condit base approxim formul effici algorithm solv squar problem appli scale formul larg data set addit exist regular techniqu squar incorpor model improv gener perform conduct experi largescal benchmark data set experiment result show propos hypergraph spectral learn formul effect captur high order relat multilabel problem result approxim formul effici origin keep competit classif perform identifi biolog relev gene multipl heterogen data sourc select gene differenti express critic biolog process major challeng post arrai analysi recent develop bioinformat made data sourc mrna mirna express profil biolog pathwai gene annot effici effect integr multipl data sourc help enrich knowledg involv sampl gene select gene bear signific biolog relev work studi problem multi sourc gene select multipl heterogen data sourc data set select gene express profil integr inform data sourc investig effect emploi inform contain multipl data sourc extract intrins global geometr pattern covari analysi gene select design conduct experi systemat compar propos approach repres method term statist biolog signific show efficaci potenti propos approach promis find multi dlabel multipl kernel learn present multi label multipl kernel learn mkl formul data embed low dimension space direct instancelabel correl encod hypergraph formul problem kernel induc featur space propos learn kernel matrix linear combin collect kernel matric mkl framework propos learn formul lead smooth min max problem cast semi infinit linear program silp propos approxim formul guarante error bound involv unconstrain convex optim problem addit show object function approxim formul differenti lipschitz continu gradient exist method emploi comput optim solut effici appli propos formul autom annot drosophila gene express pattern imag promis result report comparison repres algorithm adapt diffus kernel learn biolog network protein function predict autom annot drosophila gene express pattern control vocabulari kernel uncorrel regular discrimin analysi theoret comput studi gener linear discrimin analysi unifi framework effici model select adapt appear base face recognit comment complet character famili solut gener fisher criterion multi dclass discrimin kernel learn convex program regular kernel discrimin analysi rkda perform linear discrimin analysi featur space kernel trick perform depend select kernel paper problem multipl kernel learn mkl rkda optim kernel matrix obtain linear combin pre kernel matric show kernel learn problem rkda formul convex program show problem formul semidefinit program sdp base equival relationship rkda squar problem binari class case propos convex quadrat constrain quadrat program qcqp formul kernel learn rkda semi infinit linear program silp formul deriv improv effici extend formul multi class case base kei result establish paper multi class rkda kernel learn problem decompos set binari class kernel learn problem constrain share common kernel base decomposit properti sdp formul propos multi class case lead natur qcqp silp formul perform rkda depend regular paramet show paramet optim joint framework kernel extens experi conduct analyz connect algorithm discuss evalu ident represent facial express featur distribut interest point detect imbal orient select development stage annot drosophila gene express pattern imag entir solut path lda adapt distanc metric learn cluster good distanc metric crucial data mine task learn metric unsupervis set metric learn algorithm project observ data lowdimension manifold geometr relationship pairwis distanc preserv extend nonlinear case appli kernel trick emb data featur space kernel function comput dot product data point featur space paper propos unsupervis nonlinear adapt metric learn algorithm call naml perform cluster distanc metric learn simultan naml map data high dimension space kernel function appli linear project find low dimension manifold separ data maxim final perform cluster low dimension space perform naml depend select kernel function project show joint kernel learn dimension reduct cluster formul trace maxim problem solv iter procedur em framework experiment result demonstr efficaci propos algorithm bias manifold embed framework person dindepend head pose estim integr global local structur squar framework dimension reduct linear discrimin analysi lda popular statist approach dimension reduct lda captur global geometr structur data simultan maxim class distanc minim class distanc local geometr structur recent shown effect dimension reduct paper dimension reduct algorithm propos integr global local structur main contribut paper includ present squar formul dimension reduct facil integr global local structur design effici model select scheme optim integr balanc tradeoff global local structur present detail theoret analysi intrins relationship propos framework lda extens experiment studi benchmark data set show propos integr framework competit tradit dimension reduct algorithm global local structur squar linear discrimin analysi linear discrimin analysi lda method dimension reduct classif lda binaryclass case shown equival linear regress class label output impli lda binari class classif formul squar problem previou studi shown relationship multivari linear regress lda multi class case studi show multivari linear regress specif class indic matrix output appli preprocess step lda directli cast lda squar problem challeng multi class case paper formul multivari linear regress propos equival relationship propos squar formul lda multi class classif rigor establish mild condit shown empir hold applic involv high dimension data lda extens base equival relationship discuss discrimin kernel regular paramet learn semidefinit program regular kernel discrimin analysi rkda perform linear discrimin analysi featur space kernel trick perform rkda depend select kernel paper problem learn optim kernel convex set kernel show kernel learn problem formul semidefinit program sdp binari class case extend sdp formul multi class case base kei result establish paper multi class kernel learn problem decompos set binari class kernel learn problem addit propos approxim scheme reduc comput complex multi class sdp formul perform rkda depend regular paramet show learn automat framework experiment result benchmark data set demonstr efficaci propos sdp formul nonlinear adapt distanc metric learn cluster good distanc metric crucial data mine task learn metric unsupervis set metric learn algorithm project observ data lowdimension manifold geometr relationship pairwis distanc preserv extend nonlinear case appli kernel trick emb data featur space kernel function comput dot product data point featur space paper propos unsupervis nonlinear adapt metric learn algorithm call naml perform cluster distanc metric learn simultan naml map data high dimension space kernel function appli linear project find low dimension manifold separ data maxim final perform cluster low dimension space perform naml depend select kernel function project show joint kernel learn dimension reduct cluster formul trace maxim problem solv iter procedur em framework experiment result demonstr efficaci propos algorithm learn kernel matrix discrimin analysi quadrat constrain quadrat program kernel function plai central role kernel method paper autom learn kernel matrix convex combin pre kernel matric regular kernel discrimin analysi rkda perform linear discrimin analysi featur space kernel trick previou studi shown kernel learn problem formul semidefinit program sdp computation expens recent advanc interior point method base equival relationship rkda squar problem binari class case propos quadrat constrain quadrat program qcqp formul kernel learn problem solv effici sdp exist work kernel learn deal binari class problem show qcqp formul extend natur multi class case experiment result binari class multiclass benchmark data set show efficaci propos qcqp formul discrimin dmean cluster present theoret studi discrimin cluster framework recent propos simultan subspac select linear discrimin analysi lda cluster empir result shown favor perform comparison popular cluster algorithm inher relationship subspac select cluster framework understood due iter natur algorithm show paper iter subspac select cluster equival kernel mean specif kernel gram matrix signific insight natur subspac select procedur base equival relationship propos discrimin mean diskmean algorithm simultan lda subspac select cluster automat paramet estim procedur present nonlinear extens diskmean kernel show learn kernel matrix convex set pre kernel matric incorpor cluster formul connect diskmean cluster algorithm analyz present theori algorithm evalu experi collect benchmark data set intern confer machin learn applic effici model select regular linear discrimin analysi classic linear discrimin analysi lda applic small sampl size problem due singular scatter matric involv regular lda rlda simpl strategi overcom singular problem appli regular term commonli estim cross valid set candid cross valid computation prohibit candid set larg effici algorithm rlda present comput optim transform rlda larg set paramet candid approxim cost run rlda small number time facilit effici model select rlda intrins relationship rlda uncorrel lda ulda recent propos dimens effici model select regular linear discrimin analysi classic linear discrimin analysi lda applic small sampl size problem due singular scatter matric involv regular lda rlda simpl strategi overcom singular problem appli regular term commonli estim cross valid set candid cross valid computation prohibit candid set larg effici algorithm rlda present comput optim transform rlda larg set paramet candid approxim cost run rlda small number time facilit effici model select rlda intrins relationship rlda uncorrel lda ulda recent propos dimens kernel uncorrel orthogon discrimin analysi unifi approach kernel algorithm recent propos nonlinear discrimin analysi method address singular problem high dimension featur space attent focus properti result discrimin vector featur vector reduc dimension space paper present formul kernel discrimin analysi propos formul includ special case kernel uncorrel discrimin analysi kuda kernel orthogon discrimin analysi koda featur vector kuda uncorrel discrimin vector koda orthogon featur space present theoret deriv propos kuda koda algorithm experiment result show kuda koda competit comparison nonlinear discrimin algorithm term classif accuraci null space versu orthogon linear discrimin analysi dimension reduct import pre process step applic linear discrimin analysi lda method supervis dimension reduct classic lda formul requir nonsingular scatter matric involv undersampl problem data dimens larger sampl size scatter matric singular classic lda fail extens includ null space base lda nlda orthogon lda olda propos past overcom problem paper present comput theoret analysi nlda olda main result show mild condit hold applic involv high dimension data nlda equival olda perform extens experi type data result consist theoret analysi present analysi experiment result provid insight lda base algorithm adapt appear base face recognit regular discrimin analysi high dimension low sampl size data linear quadrat discrimin analysi wide area data mine machin learn bioinformat friedman propos compromis linear quadrat discrimin analysi call regular discrimin analysi rda shown flexibl deal class distribut rda appli regular techniqu emploi regular paramet chosen jointli maxim classif perform optim pair paramet commonli estim crossvalid set candid pair computation prohibit high dimension data candid set larg limit applic rda low dimension data paper algorithm rda present high dimension data estim optim regular paramet larg set paramet candid effici experi varieti dataset confirm claim theoret estim effici show properli chosen pair regular paramet rda perform favor classif comparison exist classif method cpm covari dpreserv project method multipl structur align consensu identif protein featur reduct gener uncorrel linear discrimin analysi comput theoret analysi null space orthogon linear discrimin analysi dimension reduct import pre process step applic linear discrimin analysi lda classic statist approach supervis dimension reduct aim maxim ratio class distanc class distanc maxim class discrimin wide applic classic lda formul requir nonsingular scatter matric involv undersampl problem data dimension larger sampl size scatter matric singular classic lda fail extens includ null space lda nlda orthogon lda olda propos past overcom problem nlda aim maxim class distanc null space class scatter matrix olda comput set orthogon discrimin vector simultan diagon scatter matric appli successfulli applic spatial interest pixel sip low dlevel featur visual media data import applic biggest challeng visual media data extrem high dimension present compar studi spatial interest pixel sip includ sip miner harri luca kanad extract consid import step reduc dimension visual media data extens case studi shown us sip lowlevel featur visual media data class preserv dimens reduct algorithm gsvd appli reduc dimens featur vector base sip experi show superior pca ddimension singular decomposit map imag idr fqr increment dimens reduct algorithm qr decomposit dimens reduct critic databas data mine applic cient storag retriev high dimension data literatur dimens reduct scheme linear discrimin analysi lda common aspect previous propos lda base algorithm singular decomposit svd due di culti design increment solut eigenvalu problem product scatter matric lda work design increment lda algorithm paper propos lda base increment dimens reduct algorithm call idr qr appli qr decomposit svd unlik lda base algorithm algorithm requir data matrix main memori desir larg data set importantli insert data item idr qr algorithm constrain comput cost appli cient qr updat techniqu final evalu ectiv idr qr algorithm term classif accuraci reduc dimension space experi real world data set reveal accuraci achiev idr qr algorithm close accuraci achiev lda base algorithm idr qr algorithm comput cost data item dynam insert dstage linear discrimin analysi qr ddecomposit abstract linear discrimin analysi lda method featur extract dimens reduct wide applic involv high dimension data imag text classif intrins limit classic lda call singular problem fail scatter matric singular lda extens propos past overcom singular problem extens pca lda stage method receiv attent pca lda lda stage preced intermedi dimens reduct stage princip compon analysi pca previou lda extens computation expens scalabl due singular decomposit gener singular decomposit paper propos stage lda method lda qr aim overcom singular problem classic lda achiev effici scalabl simultan kei differ lda qr pca lda li stage lda qr appli qr decomposit small matrix involv class centroid pca lda appli pca total scatter matrix involv train data point justifi propos algorithm show relationship lda qr previou lda method extens experi face imag text document present show effect propos algorithm index term linear discrimin analysi dimens reduct qr decomposit classif character famili algorithm gener discrimin analysi undersampl problem gener discrimin analysi base optim criterion present criterion extend optim criteria classic linear discrimin analysi lda scatter matric singular effici algorithm optim problem present solut propos criterion form famili algorithm gener lda character close form studi specif algorithm uncorrel lda ulda orthogon lda olda ulda previous propos featur extract dimens reduct olda algorithm propos paper featur reduc space ulda uncorrel discrimin vector olda orthogon conduct compar studi varieti real world data set evalu ulda olda term classif accuraci gener low rank approxim matric problem comput low rank approxim matric novelti approach low rank approxim sequenc matric unlik linear project method face recognit unconstrain illumin compar studi face recognit unconstrain illumin fr receiv extens studi exist illumin subspac present studi comparison princip compon analysi pca subspac linear discrimin analysi lda problem pca subspac lda linear project method character trace optim scatter matric gener linear project method deriv appli specif matrix analysi techniqu specif scatter matric optim criterion linear project method propos recent gener singular decomposit qr decomposit matrix analysi techniqu paper present compar studi linear project method fr involv multiresolut analysi studi compar studi expect give comprehens view perform linear project method fr problem featur extract gener uncorrel linear discrimin analysi gener low rank approxim matric problem comput low rank approxim matric novelti approach low rank approxim sequenc matric unlik gpca effici dimens reduct scheme imag compress retriev idr fqr increment dimens reduct algorithm qr decomposit dimens reduct critic databas data mine applic cient storag retriev high dimension data literatur dimens reduct scheme linear discrimin analysi lda common aspect previous propos lda base algorithm singular decomposit svd due di culti design increment solut eigenvalu problem product scatter matric lda work design increment lda algorithm paper propos lda base increment dimens reduct algorithm call idr qr appli qr decomposit svd unlik lda base algorithm algorithm requir data matrix main memori desir larg data set importantli insert data item idr qr algorithm constrain comput cost appli cient qr updat techniqu final evalu ectiv idr qr algorithm term classif accuraci reduc dimension space experi real world data set reveal accuraci achiev idr qr algorithm close accuraci achiev lda base algorithm idr qr algorithm comput cost data item dynam insert effici kernel discrimin analysi qr decomposit ddimension linear discrimin analysi human gait effect biometr sourc human identif visual surveil human gait recognit hot topic recent research elaps time problem infanc receiv poor perform paper introduc discrimin analysi method improv perform model inherit merit tensor rank analysi handl small sampl size problem natur linear discrimin analysi optim classif dlda datr benefit method converg train procedur mean util practic applic base lot experi elaps time problem human gait recognit method demonstr significantli outperform exist appear base method principl compon analysi linear discrimin analysi tensor rank analysi optim criterion gener discrimin analysi undersampl problem abstract optim criterion present discrimin analysi criterion extend optim criteria classic linear discrimin analysi lda pseudoinvers scatter matric singular applic rel size data dimens sampl size overcom limit classic lda optim problem solv analyt appli gener singular decomposit gsvd techniqu pseudoinvers suggest undersampl problem past data dimens exce number data point criterion propos paper theoret justif procedur approxim algorithm gsvd base approach present reduc comput complex find subclust cluster centroid captur structur cluster reduc problem yield smaller matric gsvd appli effici experi text data dimens show approxim algorithm produc result close produc exact algorithm index term classif cluster dimens reduct gener singular decomposit linear discrimin analysi text mine uncorrel discrimin analysi tissu classif gene express data abstract classif tissu sampl base gene express data import problem medic diagnosi diseas cancer gene express data number gene high thousand compar number data sampl ten low hundr data dimens larg compar number data point data undersampl cope perform accuraci problem high dimension commonplac appli preprocess step transform data space significantli lower dimens limit loss inform present origin data linear discrimin analysi lda techniqu dimens reduct featur extract applic undersampl data due singular problem matric underli represent paper present dimens reduct featur extract scheme call uncorrel linear discrimin analysi ulda undersampl problem illustr util gene express data ulda emploi gener singular decomposit method handl undersampl data featur produc transform space uncorrel make attract gene express data properti ulda establish rigor extens experiment result gene express data present illustr effect classifi tissu sampl result provid compar studi state art classif method gene express data set index term microarrai data analysi discrimin analysi gener singular decomposit classif pairwis protein structur align base orient dindepend backbon represent approxim multipl protein structur align sum dof dpair distanc algorithm present comput multipl structur align set protein gener consensu pseudo protein set algorithm heurist comput approxim optim multipl structur align minim sum pairwis distanc protein structur algorithm choos input protein initi consensu comput correspond protein structur repres set unit vector approach analog center star method multipl sequenc align correspond set rotat matric optim correspond deriv align structur deriv consensu process iter sum pairwis distanc converg comput optim rotat iter process make current consensu gener simultan approach base interest result sum pairwis distanc repres compactli distanc consensu experiment result protein famili present show algorithm converg rapidli kei word structur align center star method dynam program consensu protein correspond rotat matrix frobeniu norm singular decomposit lda fqr effici effect dimens reduct algorithm theoret foundat spatial interest pixel sip low dlevel featur visual media data import applic biggest challeng visual media data extrem high dimension present compar studi spatial interest pixel sip includ sip miner harri luca kanad extract consid import step reduc dimension visual media data extens case studi shown us sip lowlevel featur visual media data class preserv dimens reduct algorithm gsvd appli reduc dimens featur vector base sip experi show superior pca optim criterion gener discrimin analysi undersampl problem pairwis protein structur align base orient dindepend represent backbon geometri determin structur similar protein import problem identifi function evolutionari relationship paper algorithm propos align protein structur protein backbon algorithm nd rigid motion backbon larg substructur match algorithm represent backbon independ rel orient space appli dynam program represent comput initi align ned iter experi algorithm competit algorithm dali lock approxim minimum energi curv effici approxim minimum energi curv interpolatori constraint 